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Abstract 

It has been argued that Nekrasov's partition function gives the generating func- 
tion of refined BPS state counting in the compactification of M theory on local 
Calabi-Yau spaces. We show that a refined version of the topological vertex we 



previously proposed (hep-th/0502061 ) is a building block of Nekrasov's partition 



function with two equivariant parameters. Compared with another refined topo- 
logical vertex by Iqbal, Kozcaz and Vafa (hep-th/0701156), our refined vertex is 
expressed entirely in terms of the specialization of the Macdonald symmetric func- 
tions which is related to the equivariant character of the Hilbert scheme of points 
on C^. We provide diagrammatic rules for computing the partition function from 
the web diagrams appearing in geometric engineering of Yang-Mills theory with 
eight supercharges. Our refined vertex has a simple transformation law under the 
flop operation of the diagram, which suggests that homological invariants of the 
Hopf link are related to the Macdonald functions. 



1 Introduction 



The problem of instanton counting is one of the important aspects of nonperturbative 
dynamics in gauge and string theory. The resuh is encoded in the partition function of 
topological gauge and string theories, which is often computed exactly by the duality 
and/or the localization principle. A celebrated example in gauge theory is Nekrasov's 
partition function Z^eki^i, ae, A), which reproduces the Seiberg-Witten prepotential from 
the microscopic viewpoint of equivariant integration over the instanton moduli space [Ij. 
On the string theory side, the topological vertex CaiA2A3(?) is constructed based on the 
geometric transition, which is a duality of topological closed string to the Chern-Simons 
theory [21 13]. The topological vertex provides a building block of all genus topological 
string amplitudes on local toric Calabi-Yau 3-fold. It is amusing that these two instanton 
counting problems are actually related in an appropriate setup, which is expected from 
geometric engineering [U [5] . 

To compute the integration over the instanton moduli space of SU (N) gauge theory, 
Nekrasov considered the toric action on M'* ^ 9 {zi,Z2) (e*'^^zi, e*'^^Z2), which 
induces the toric action on the moduli space of framed instantons. By the localization 
theorem, the integral becomes a sum over the contributions from each fixed point of the 
toric action, which is labeled by the set of Young diagrams (the "colored" partitions) 
{A^}^]^, (A^,i > \£^2 > ■ ■ • > ^e,i > ^e,i+i >■■■)• As a consequence we obtain Nekrasov's 
partition function [6]: 

J^2N\X\ 

2'Arefc(ei,e2,a^,A) = -, (1.1) 

{A,} [L,f3=l'^a,l3 [(^1, (^2, ae) 

where |A| = Xl^Li Y.i<i \i and 



= n (-^A,(s)ei + {axSs) + 1)^2 + ap- a„) 



seAo 



W ((£a. it) + l)ei - ax, (t)e2 + ap - a„) . (1.2) 

The parameter A of instanton expansion is introduced as a dynamical scale in the renor- 
malization. The vacuum expectation values of the scalar fields in the vector multiplets 
are Oq,, 1 < a < A^. Mathematically they are equivariant parameters for the action of 
maximal torus on the gauge group. We denote the leg length and the arm length at 
s = {hi) with respect to the Young diagram A by = AJ — i, and a\{i,j) = \i — j, 

respectively. The relation to the (topological) string theory becomes transparent, if we 
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consider a five- dimensional ( "trigonometric" , or i^-theoretic) lift of the partition function 

a,l3 



by promoting the factors n^ffi^i, ^2, cli) in the denominator to 



SSAa *£A^ 

(1.3) 

where (5', t) := (e^^,e~^^) and := e~"'°' . We can show that, when q = t = e~^% 
Nekrasov's partition function is nothing but the topological string amplitude on an ap- 
propriate local toric Calabi-Yau manifold [7] |12j. 

All genus topological string amplitude on local Calabi-Yau 3-fold can be computed 
by a diagrammatic rule, in terms of the topological vertex: 

CM^l) = q'^sx{qnj2'>^/v(^'^^'>-^yvi<l'''') , (1-4) 

V 

where sx/^{x) is the (skew) Schur function and q'^^^ means the substitution xi := g'^»~*+5. 
The partition is defined by the transpose of the corresponding Young diagram. The 
definition of k(A) is given in Appendix eI^. Then a natural question is: For generic 
parameters (ei, €2), can we obtain Z^eki^i, o-e, A) in a similar manner by generalizing the 
topological vertex C^xu^q)"^ This is the problem of constructing a refined topological 
vertex. An answer to this question has been given by Iqbal, Kozcaz and Vafa [T3]. The 
refined topological vertex they proposed is 

(l-j ""' t'^P,4t-';g,t)J2{j) '" ■V'/nil-'t-'K/nit-'"'!-') ■ (1-5) 

As before, qH^ etc. means the specialization Xi := q-^H^~\ On the other hand, before 
the proposal in [13] we had introduced the following vertex in [14^ . 

I ?7 1 — I '-^ I 

/,(g,t)-V,(t^;g,t)^(|) ' iP^./,.i-t'\^-t,q)P,/,{q't^-q,t). (1.6) 

V 

It is convenient to introduce the conjug ate vertex C^^iq, t) := C^^vx^^^ {t, q){-l)M+\M+H. 
Note that the conjugation involves the exchange of t and q. The refined vertex C^JJ^^\t, q) 



^Our notations for partitions arc summarized in Appendix E. 

^We have slightly changed the original definition in '14} by improving the framing factor. 



2 



partly employs the Macdonald function Px{x;q,t) [T5], but there still remain the skew 
Schur functionjfl. Compared with it, our proposal eliminates the skew Schur func- 
tions completely and the vertex is expressed in terms of the (skew) Macdonald function 
Px/^{x; q,t). The price for this elimination is that we have to introduce the involution l 
on the algebra of symmetric functions defined by = —Pn, where Pn{x) '■= Ylili ^7 
the power sum function. Since {Pn{x)}'^=i forms a "multiplicative" basis of the algebra 
of the symmetric functions, the involution l is uniquely defined by the above relation. Fi- 
nally fx {q,t) := (— l)l^lg^~t 2 is the framing factor proposed recently by Taki |16j . 
The refined vertex CjJJx^\t, q) has a nice interpretation as the counting of "unisotropic" 
plane partitions, or by statistical mechanics of the melting crystal model (TTJ [131 [H] • Al- 
though the relation of our vertex to such a statistical model is unclear, our vertex is more 
symmetric than C*^^^^'*(t, g), since we have replaced all the (skew) Schur functions in the 
topological vertex by the (skew) Macdonald functions. However, it seems impossible to 
make C^x^{q,t) completely symmetric under the cyclic permutation of partitions. 

In [T3] it is claimed (see also the arguments in [16]) that one can reproduce Nekrasov's 
partition function from the refined topological vertex C'^^^^'*(t, g). As we mentioned 
already in [H], our vertex (11. 6p also reproduces the SU (N) Nekrasov's partition function, 
and in this article we will show it concretely. Though C^^Jx^\t,q) and Cf^x'^{q,t) are 
different, they give the same result as long as we put trivial representations to external 
edges, which is the case when we compute Nekrasov's partition function by the method 
of topological vertex. The Schur functions and the Macdonald functions are two different 
basis of the space of symmetric functions and hence they satisfy the Cauchy formulas 
of the same type (see section 5.2 and Appendix B). This is a technical reason why 
C^Jx^\t,q) and C^x^{q,t) give the same result after taking the summation over the 
partitions attached to internal edges. We should emphasize that, throughout this article 
except for appendix D, q and t are treated as formal parameters and we do not use any 
asymptotic computations such as lYmM^ool^ = otherwise stated. 

In this paper we show that the refined topological vertex gives a building block of 
the i^'-theoretic lift of Nekrasov's partition function. We would like to point out the 
following possible application. It has been argued Nekrasov's partition function gives the 
generating function of refined BPS state counting in the compactification of M theory on 
local Calabi-Yau spaces [IIl|lH[13]. As far as we know (see appendix C, for examples), 
the refined BPS state counting always gives integers, which is a refined version of the 
conjecture of the integrality of the Gopakumar-Vafa invariants [19]-[2T]. Recently the 



^However, this implies a nice interpretation in terms of plane partitions. 
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conjecture has been proven for local toric Calabi-Yau 3-folds [22]^[2l]. The existence 
of the topological vertex is one of the important ingredients in the proofs. Hence one 
may expect that the refined vertex is helpful in proving the integrality of the refined 
Gopakumar-Vafa invariants for the local toric case. Macdonald functions are also related 
to g-deformed Virasoro and W algebras [25]. We hope these quantum groups play an 
important role in topological string theory and Yang-Mills theory. 

The paper is organized as follows: In section 2 we introduce the ii'-theoretic lift of 
Nekrasov's partition function following a mathematical formulation in [26l [27] . The K- 
theoretic lift allows the Chern-Simons coupling m G Z and we find that the framing 
factor fx {q, t) of the refined topological vertex arises naturally from the m dependence 
of the partition function. We also examine the symmetry of the partition function under 
ri : {q,t) — > {q~^,t~^) and : (g, t) — > {t,q). This is a necessary condition for the 
fT-theoretic lift to be interpreted as a character of Spin{A) = SU{2)l x SU{2)r. In 
section 3 we review the idea of geometric engineering. When we compute the partition 
function using the refined topological vertex as a building block, we will fix a preferred 
direction, for which we mainly choose the horizontal left arrow (—1, 0) in this paper. The 
(dual) toric diagram in geometric engineering has a feature in which we can arrange the 
diagram so that each vertex has a unique edge with the preferred direction. To emphasize 
the fact that we fix the preferred direction of the diagram, we will call it web diagram 
in the following. We define our refined topological vertex in section 4. The gluing rules 
of the vertex for computing the partition function are also provided. In section 5 we 
consider four-point functions obtained by gluing two refined vertices. We show that the 
four-point function enjoys a rather simple transformation law under the flop operation 
of the web diagram. Based on this transformation law we argue a possible relation of 
our refined vertex to homological invariants of the Hopf link [28l [29] . In section 6 we 
discuss one-loop diagrams with some examples. Finally, we present several examples of 
the computation of the partition function in sections 7-9. In appendix A we explain the 
equivalence between several expressions of the Nekrasov formula. In appendix B we give 
a definition of the Macdonald sjTiimetric functions and collect several useful formulas. 
We present examples of the refined BPS state counting in appendix C. In appendix D 
we remark that our refined topological vertex can be expressed in terms of the g-Dunkl 
operator. Appendix E gives a list of notations and some identities for partitions used in 
this paper. 

The following notations are used through this article, q and t are formal parameters 
otherwise stated. Let A be a Young diagram, i.e. a partition A = (Ai, A2, ■ ■ ■ )? which is 



4 



a sequence of nonnegative integers such that > Aj+i and |A| = J2i Aj < oo. A^ is its 
conjugate (dual) diagram. ^(A) = X( is the length and |A| = the weight. For 

each square s = in A, 

a{s):=Xi-j, a'(s):=j-l, 

£(s):=A;-^, i'{s):=z-l, (1.7) 

are the arm length, arm colength, leg length and leg colength, respectiveljfl. Let p„(x) = 
Xli^i ^7 t>e the power sum function in the set of variables x = {xi,X2, ■ ■ ■ )• If 1*^"^! < 1; 
the variable qH^ stands for Xi = g'^'t^"*. But for all t G C, we define 

p„(cgV) := c" y (g"^- - l)^^"*) + q,t,ce C, 



oo 



1 T n 

p„(cgV, cLt^O := V (g"^» - l)^^^-*) + c"^ —, g, t, c, L G C 

t2 —t 2 

4 = 1 

TV 

= c"^g"^'r(i-*), g,t,cGC, L = t-^,NeN. (1.8) 

All symmetric functions in this article are treated as polynomials in the power sum 
symmetric functions {pi,P2, ■ ■ ■)• Finally, we often use u := {qt)^ and v := {q/t)^. 



2 Structure of Nekrasov's Partition Function 

2.1 Partition function with Chern- Simons coupling 

The five- dimensional lift of Nekrasov's partition function of SU {N^) theory is given by 
the summation over the set of Nc Young diagrams (or colored partitions) {Xa}^!^, as 
follows: 



Z'-*(ei,e2;a,,A) = 5^ 



The product in the denominator is the equivariant Euler character of the tangent space 
to the instanton moduli space M{Nc, k) at a fixed point of the toric action, which is 
labeled by {Xa}a=i with |A| = A; . Each factor is given by 

(2.1) 



"'in [13] the definitions of the arm length and the leg length are exchanged. 
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The product 11^/3=1 ^i^i3^ (^i) ^2; cia) consists of 2Nck factors, which agree to the complex 
dimensions of M{Nc, k). In the five- dimensional lift is mathematically identified 

as the i^'-theoretic lift and it is computed as follows: 

Zrnei,e2;a„,A) 

k=0 i 

|A| 

e 2V 



= E Vr Mi>^^}f '—■explmJ2J2(^»-ns)e^-a'is)e2)] , (2.2) 

{Ac} lla,/3^V/3 iei,e2;a„j \ „ ^^^^ y 

where M{Nc, k) is the framed moduli space of rank N^. torsion free sheaves E on with 
C2{E) = k. The line bundle C over M{Nc, k) is defined by 

C := det [R\p2).{S ® (pi)*Op2(-£oo))] , (2.3) 

where S is the universal sheaf on x M{Nc, k) and pi^2 is the projection to the first or 
the second component. Physically Z^^^^^ is the instanton part of the partition function of 
SU {Nc) gauge theory on x with eight supercharges, and the power m G Z of the 
line bundle C is identified as the coefficient of the five-dimensional Chern-Simons term 
[30]. 

Let (g, t) := (e'^e"'^), ea := e~°'"' and Qa,f3 ■= Ga/e/s^ then Z^''*(ei, €2; aa. A) is 
written as 

(ei,---,e^^,A,g,t)-2^ — , (2-4) 

{A„} 1 la,/3=l ^^-^'^A/S 9, 

with t> := {q/t)2 and 

iVAcA, (Qa,/3; q, t) := N^^^^ (ei, €2; a«) 

where |A| is the number of boxes of A and 

fxiq,t) := lli~l)q'^'^-'^~'n-''(^^~'^ = J] (-l)g^-^'+^r^^+^-^ (2.6) 
sex («j')eA 



^The different conventions {q,t) := (e "^^,6*^^) and Qp^a ■— are afso used in tlie literature. 
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is the framing factor § which has been proposed by Taki [16] . This is nothing but the m 
dependent (g, t) factor of the partition function. Note that the framing factor satisfies 
the following symmetry: 

fx (g, t) = fx {q-\ t-') ' = /,v (t, q)-' . (2.7) 
1 1 

Let u = {qt)2 and v = (q/t)^. We have the following six equivalent expressions of 

Nx^{Q;q,ty. 

Proposition. 



N,,{Q;q,t)= n {l-Qq'^'H^^-'^') J] (^"^^ 



(2.8) 
(2.9) 



Nxf. (Q; g, t) = Ho [-v-'Q g V, t'^'g") /Hq (-tr^Q t^, qp) , (2.10) 
iV;,^(g;g,t) = no(-^;-^gr"'g-^ g-^r^) /Hq ^-^Q r^) , (2.11) 



iV,^ (g; g, t) = n (g g V, g-^t-^; g, t) /U {Q t^, r"- g, t) 



where 



no(-X, ?/) := exp i - ^ -Pn{,x)pn{y) \ = JJ(1 - XiVj), 
I n>0 j i,j 



U{vx,y]q,t) :=exp <^ ^ 



. n>0 



^ n n 
1 t2 - 

n n 

?T- g2 — g~ 2 



Pn(a;)p„(i/) 



{uxiyj;q)c 



n 



ni^a^XiVfq ^)c 



(2.12) 



iV,^ (g; g, t) = n (g t'^'g", t-^'g-^; g-^) /H (g g^ g""; g-^) . (2.13) 



(2.14) 

.^ , |g|<l 
vx^yj , gj 

^'^ (2.15) 

|g-i| < 1. 



Here {x;q)oo is the g-shifted factorial (x; g)c 
Ig'^l, < 1, fl2J0D is written as 



nfcez>o(-'- ~ q^'x). Note that when 



oo 

II (l -Qq^^-H^^-'+'^ I (1 - gt^-^g-^) . 



(2.16) 



In terms of ||A||2 Af = 2E.eA(«(s) + \)^ this is A (g,t) - 
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and also when |g| < 1, fl2.12p is 

The equivalence of these six expressions are proved in app. A. The first formula, fl2.8p . is 
given by [6], and fl2Anil -( l212ll by [31]. In this article, we mainly use (El]) and (l230ll . 

2.2 Another form of the partition function 

Let us transform Nekrasov's partition function Z^^'^^ so that it becomes transparent, and 
compare it with the amplitude constructed by the method of topological vertex. Using 
(EHD, i.e. 

n ^^^'^ n Q'^'^^'^ = n n (2-i8) 

we can show, from (12.80 . that 

N,x {Q-'; q, t) = N,, {v'Q; q, t) Q-^'^-^^^f, {q, t) /fx {q, t) 

= iV^v.v [Q- 1, q) {vQ)-\^\-\^\f, {q, t) /f, (g, t) . (2.19) 

Here we use (12.331) . Hence we have 

Nc Nc 

n ^-^/3^- ^) = n ^-^/5^^-^ ('^a,/?; q) 

a<(3 a<(3 

Nc / a-1 Nc-l \ ^1-^"' 

xji h"-^n^i.+i n ^.Vi /A.(^,t)-"="^"-\(2.2o) 

a=l \ /3=1 /3=a / 

and thus Nekrasov's formula (12. 4p is rewritten as 

rr^'= A l^-l A (a 4.\Nc-m-2a+l 
^inst _ \.\.a=l^^a,rn J\a\Hi^) /2 21 ^ 

" ~ x.h.c ni^</5 ^^"^^ ^) ^^''^^"^ ^) n^ii A^A.A. (1; g, t) ' 

where 

a-1 Nc-l 

Km := v-'K'^^ (-ej- n n ^^m- (2-22) 

/3=1 /3=a 

For example, for SU{2) theory we have 

^in.t ^ V- (^^-^A^Qh)''' (-ei)-l^^l(-e2)-l^^'/A, (g,t)^-/., (g,t)-^- 

" a'tX ^AiA.(Q/f;g,t)iVA2VAiv(QH;t,g)iVAiAi(l;g,t)iV;,,Ajl;g,t) 
_ ^ (t;-^A^gH)'" (-ei)-'"l^-l(-e2)-l^-l/A, (g^t)^-/,, (g,t)-^- 

iVAai(i;g,t)iVA2A2(i;g,^) 

8 



Uoi-v-^Qng^HP, t^'^''qP)no{-vQHt^'"qP, q^'tP) ' ^ ' ' 

where 

^^AA (1; g, t) = n (1 - (l - g-^W-^t-^^^)) , (2.24) 

sex 

and Qh = Q12. It is this form of Nekrasov's partition function that is obtained from the 
refined topological vertex with formulas of the Macdonald functions. 

2.3 Symmetry as a character of Spin{A) 

If Nekrasov's partition function gives the generating function of refined BPS state count- 
ing in the compactification of M theory on local Calabi-Yau spaces, it has to be a 
character of Spin{A) ~ SU{2)l x SU{2)r, since the spin of massive BPS particle in five 
dimensions is a representation of Spin{A). In general, if a function f{u, v) in two variables 
{u, v) is invariant under both u -—>■ u^^ and v it is a linear combination of Spin{4:) 

characters 

f{t,q)= a^si^,sn)XsAu)XsR{v) , (2.25) 

where 



^n+l ^— n— 1 



Xniz) := 2" + + ■ ■ ■ + + = , (2.26) 

is the character of the irreducible representation of SU{2) with spin n/2. Hence, if the 
fc-instanton part Z^''\q, t) of the partition function is invariant under the transformations 
ri : {q,t) iq'^,^'^) and : (g,t) it,q)- Z^''\q,t) is expanded as 

Z('^\q,t) = J2 4lsn)XsA^)Xs,iv) , (2.27) 

{sl,sr) 

with rational coefficients ^^(sls^)- Recall that u = ^/qt and v = \fqjt. Actually we 
will find an appropriate scaling of Z^^\q,t) depending on the instanton number k is 
necessary for the genuine invariance under the above transformations. If the partition 
function takes the form fl2.27p . then the /c- instanton part F^''\q,t) of the free energy is 
also a linear combination of Spin{4) characters. Furthermore, if the pole structure of the 
free energy is appropriate, we can factor out the character of the half-hypermultiplet and 
subtract the multicovering contributions to obtain the expansion of the total free energy 
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in the Gopakumar-Vafa form: 

oo 

F = \ogZ = Y,F'^'\Qp;q,t) 



k=0 



= E EE „(,,,;_ i)(„„_„„) X-,.i.Mx„.,.(..)Qg. (2.28) 

The coefficients A^-^^'^'^'^^ of the expansion fl2.28p are conjectured to be nonnegative inte- 
gers, since from the viewpoint of the Calabi-Yau compactification of M theory they are 
interpreted as multiphcities of the five-dimensional BPS particles arising from M2 branes 
wrapping on a two-cycle /? G H2{X,Z) in the Calabi-Yau 3-fold X. We have checked 
the integrality of the refined BPS state counting from the SU{2) and SU{3) partition 
functions up to instanton number 2. The result is presented in appendix C. 

Since the transformation (g, t) — > (t^^, q^^) is compensated by the transpose of colored 
partitions, we have 

n ^A„A, {Qa,^;r\q-') = n ^AVAV (Q.,/?; g, t) • (2.29) 

a,l3=l a,/3=l 

By f lCTD . we also find that 

q\Nc\x\ 
t) 

a,l3=l a,/3=l 

Therefore, we obtain 



n {Q~a!p-^i'\t-') = (I) ^ n ^A.A,(Qa,/5;g,t) . (2.30) 



j2 — - = Yl — - 

{Xc.},\X\=k lla!(3=l ^A<,A^ {Qa,(3] t \ q ^) Ua!f3=l iQa,l3] Q, t) 

1 ft\^'^^^ 



{X^},\X\=kUa^P=l^^c.xAQa!f3'^1 ^^ ^) {A,},lAl=fc na,>=l ^A.A^ (Q",/?! ^) 

(2.31) 

Thus if we can prove 

{xjjx\=k nS=i ^A„A^ {Q~^; q, t) ^xjjx\=k nS=i ^A„A^ (Qa,/?; q,t) 

the partition function .^^'^^ is invariant under both of the reflections tl and r^j. It is easy 
to see that the property fl2.32p is valid for Nc = 2, since the exchange of two partitions 
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effectively induces — » e^^. For Nc > 2 the validity of (12.321) seems nontrivial. The 
overall reflection of the roots e^^ cannot be induced by any permutation of colored 

partitions. However, we have checked by explicit computations that (12.321) is true for 
Nc = 3 and k = 1,2. This is consistent with the computation in appendix C, where we 
obtain the results of refined BPS state counting from Nekrasov's partition function of 
SU{3) gauge theory. 

The symmetry of Nekrasov's partition function with the Chern-Simons coupling can 
also be derived easily. Because of ( 1A.20I) . the factor N\^{Q]q,t) enjoys the following 
duality relations: 

iV^^ {vQ- q, t) = iV^A [v'^Q] q-\ t"') = N^-x- {v^'Q; t, q) . (2.33) 
From the expression (12.91) . we also have 

Nx, {vQ- q, t) = N,, {vQ-\ g, t) QI^I+I^I/a (g, t) / f, (q, t) . (2.34) 

Since 

Nx, {v'Q; q, t) iV^A {v'Q-'; g, t) = N^, (Q; q, t) iV^A {Q-'; g, t) (2.35) 

we find that 

N,^ {Q; q, t) iV^A (Q"'; g, t) = N,, {Q-'; q-\ r^) N^, {Q; q-\ r') 

= iVAv^v (Q-i; t, q) iV^vAv (Q; t, q) (2.36) 

Thus, Nekrasov's partition function Z'!^^ has the following symmetries: 

Z'::' (ei, ■ ■ ■ , A; g, t) = {ei\ ■ • • , e^^\ A; q-\ r') 

= ZLt(er\---,e^^\A;t,g). (2.37) 

3 Geometric Engineering and Toric Geometry 

In this section following [H O [HI [H] , we review the toric geometry that is necessary for 
geometric engineering. Geometric engineering tells how to obtain M = 2 SU{Nc) super 
Yang-Mills theory with Nf fundamental matters from type 11(A) string theory on local 
Calabi-Yau manifold Ks, the canonical bundle of a 4-cycle S. The (toric) geometry 
of the 4-cycle S can be described by the (dual) toric diagram. The prescription of the 
geometric engineering implies that the toric diagram of S has Nc horizontal internal edges 
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("color" D'b branes) and Nf horizontal external edges ("flavor" D5 branes). For example, 
the vertical distance of "color" branes represents vacuum expectation values of the 
Higgs fields or the mass of W bosons. The matter fermions are given by fundamental 
strings connecting a "color" Db brane and a "flavor" Db brane. The vertical distance of 
a "color" brane and a "flavor" Db brane represents the mass of the corresponding 
matter fermion. 

One of the properties of toric diagrams that arise from geometric engineering is that 
each vertex has a unique horizontal edge. In the following we will consider toric diagrams 
in which we specify the horizontal edges as distinguished. In the computation by the 
method of topological vertex, we cut the internal horizontal edges. Then the contribution 
of each component is given by an amplitude of "the vertex on a strip" [32]. By gluing 
these amplitudes we obtain the partition function for the local toric Calabi-Yau manifold 



In the compactification of type IIA string theory on local Calabi-Yau manifold, M = 2 
supersymmetric SU{N) gauge theory is geometrically engineered by ALE fibration of 
An -I type over the rational curve P^. The fiber consists of a chain of — 1 rational 
curves whose intersection form is given by the minus of the Cartan matrix of A^-i- The 
holomorphic 2-cycles in the fiber are in one-to-one correspondence with the positive roots 
of A]sf_i. The (dual) toric diagram takes the form of "ladder" diagram with A^ parallel 
horizontal edges. In the toric diagram the faces correspond to compact 4-cycles (divisors). 



Figure 1: Ladder diagram for SU{N) gauge theory. There are A^ + 1 possible toric 
diagrams (m = 0, ■ ■ ■ A^). 

In the ladder diagram of ALE fibration over P^, we find A^ — 1 divisors, all of which 
are P^ fibration over P^, namely the Hirzebruch surfaces. The degree of the Hirzebruch 
surface can be determined by the (relative) slopes of the vertical edges of the face. We 
will denote the Hirzebruch surface of degree n by F„. It is known that for each A^ there 




(m + 1,1) 
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are + 1 types of such geometry, which we label by m = 0, 1, ■ ■ ■ [HI [H]. The integer 
m is related to the coupling constant of five- dimensional Chern-Simons coupling [30]. Let 
us call such geometry toric SU{N)m geometry. We can characterize the toric SU{N)m 
geometry by saying that its compact 4-cycles are {F]\f^2+m,FN-4+m, ■ ■ 




Figure 2: Subdiagram of the k-th divisor of SU{N)m geometry {1 < k < N — 1) 

The Kahler parameters of SU{N)m geometry are Tg of the base space and Tp. {i = 

1, - ■ ■ ,N — 1) of the fiber which is a chain of {N — 1) P^'s. In the subdiagram of Figure 

2, the rational curves of both side edges correspond to the fiber of FAr_2A;+2, and their 
Kahler parameters are Tp^. On the other hand, if we denote the Kahler parameters of 
the upper and the lower edges by Tb^. and Tb^.^^, respectively. The difference is related 
to the degree of the Hirzebruch surface as follows: 

Tb, - Tb,^, = (N - 2k + m)Tp^ . (3.1) 

From the recursion relation (13.11) we find, if + m = 2r + 1 is odd, that 

Tb,. ■=Tb , 

r 

Tb, = Tb + Y^i^ + ^ - 2j)^F, , (1 < ^ < r - 1) , 

j=i 

i-l 

Tb, =Tb+ Y,{2j-N- m)TF^ , {r + I < i < N) , (3.2) 

j=r+l 



and if + m = 2r is even 



TBr —Tb,.+i '-—Tb , 

r-l 

Tb, =Tb + ^(AT + m - 2j)Tp, , (1 < ^ < r - 1) 



i-l 



Tb,=Tb + ^ (2j - AT - m)TF^ , {r + 2<t<N) . (3.3) 

j=r+l 



13 



In (13.21) and (13. 3p we take the first relations as initial conditions in solving (13.11) . 

From the slope of each edge in Figure 1, we can also compute the framing index by 
the rule to be explained in section 4.2. Let us denote the index of left, right, upper and 
lower edges by nL,k,nR^k,nB,k and n^.fc+i, respectively. Then we compute 

ns,fe = (m - A; + 1, 1) A (-iV + k,l) = N + m - 2k + 1, 
nL,k = {-l, 0) A{N -k -2,-1) = 1, 

nij,fc = (m-A;,l) A(-1,0) = 1. (3.4) 

Note that n^^ fc and n/j ^ are independent of k. By definition the framing index changes 
the sign, if we reverse the orientation of the edge, or replace the representation associated 
to the edge by its transpose. We will use these framing indices in the computation of the 
partition function by gluing the refined topological vertices. 

4 Refined Topological Vertex 

In [13], we defined the refined topological vertex which is written not by the Schur 
functions but by the Macdonald functions. Here we slightly modify it by improving the 
framing factor. 

4.1 Refined topological vertex 

Let Px/^{x; q,t) and {Px\Px)q,t be the Macdonald function in the infinite number of vari- 
ables X = {xi,X2, ■ ■ ■) and its scalar product, respectively, defined in Appendix B. We 
introduce an involution l acting on the power sum function Pnix) by t{pn) = —Pn- For 
example, 

oo ^ 

iPniq'tn = - - l)r(^-) - „ ^ (4.1) 

^ t2 — t 2 

Note that LPn{tP) = -Pn{tP)=Pn{t-P). 

We define a vertex V^x' as followslll 

V^x"" :=PA(t";g,t)$^^P;.v/,v(-t^"g'';t,g) P,/^{qhP; q,t)v\^\ , (4.2) 

a 

^ Although we wiU show that the Nekrasov formula is represented by our vertex V^a'^i one can also 
produce it through the following vertex without the involution i 
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where 

fAC".".')-!! /. 'wiVi - PA.(-.;';M)-n i_ '!(,)-V«(.) - 

sex ^ seA ^ 

which follows by substituting Q = into (IB. 211) . From ( IB. 201) . V^j.x'^ is rewritten as 

V.x" = Px{t';q,t)Y,^Pf^/A<l''t-^;<l,t) P,/^iqH'';q,t){P^\P^),,t 9,{q,t), (4.4) 

a 

with 

which satisfies 

9x{q,t) = gx{q-\t-') = gxAt,q)~'- (4.6) 
From (14. 4p . (IB.28P and (IB. 271) . one can show the symmetrj|§ 

9x{q,t)-'Vx.' = V.x' = V..\ (4.7) 
g,{q,trX.'' = 9u{q,tr'V,.^, (4.8) 
V.x'' = V.,\ (4.9) 

Incorporating the framing factor, we define our refined topological vertices C^x'^{q,t) 
and C^^iy{q,t) as follows: 

C,x''{q,t):=V,x''v-\''\U{q,tr' (4.10) 
= Pxit''; q, t) J2 ^P,V<r^ (-^''/; ^./<x(gV; g, t)t;l'^l-l'^l/, (g, t)"^ , 

C^Kiq, t) := C^v.v^^ {t, g)(-l)l^l+H+IH (4.11) 
= Pav i-q"; t, q) J2 ^.v/.v {-t'\P; t, q) iP^/M^t^- q, t)t;^'^l+l^l/, (g, t) . 

The lower and the upper indices correspond to the incoming and the outgoing repre- 
sentations, respectively, and the edges of the topological vertex are ordered clockwise. 
Although only the refined vertices of the above types are mainly used in this article, the 
following vertices may also be useful: 

C^xuiq, t) :=C,x''{q, tW^^^^''\ f, {q, t) U (?, t) (4.12) 
= P,(t^; g, t) ^P,V^^ P^/Aq't"; q, t^^^^^^^^ f, (g, t) , 



^ If we replace Macdonald functions Px/^(x; g, t)'s in V^a'' by "normalized" Macdonald functions 
Px/ti{x]q,t) Px/fj.{x] q, t)\/gx{Q, t)/g^{q, t), then the g factors in (|4.7p - l|4.9p disappear. Because 
Px{x; q,t)Px--y {y]t, q) — Px{x; q,t)Pxv (i/;t, q), all results in this article remain the same even if we use 
the normalized Macdonald functions Px/fj,{x; q,t). 
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C.^'ig, t) := C^\{q, t)t;-l'^'-l'^l^ (g, ty' U {q, t)'' = C^\^u^ (t, (4.13) 
= Pav (-g''; t, g) P.v/,v (-t^^g^; t, g) iP./M^tP- g, (g, t)^' . 

a 




Figure 3: Refined topological vertex: the represantation for the preferred direction, i.e. 
the middle index A, is indicated by the white arrow. 

Note that, when q = t, the topological vertex in [3] is 

c^,A.(g) = 3A(gOE SuvAq'^nU^"^'^''^'^- (4.14) 

Since S/i/o-(g^''^'') = LSf^/aiq^^^^), which follows from flA.llI) . our refined topological 
vertex limt^g C^x'^{q,t) coincides with the topological vertex C^\^v{q). It is well-known 
that in the operator formalism the Schur functions are realized in terms of free fermions. 
Although we have no fermionic realization of the Macdonald functions, they are described 
by bosons as shown in [33]. Our refined topological vertex has a bosonic realization by 
using that for the Macdonald functions. 



4.2 Gluing rules 

Here we show our gluing rules for constructing the partition function from a web diagram. 
Let us consider a graph with trivalent vertices and edges. Each edge is associated with 
an integer vector v = (fi,f2) G Hence the trivalent vertex with edges indexed by 
{i,j,k) in the counterclockwise ordering is associated with a triplet of integer vectors 
{vi,Vj,Vk). If we choose these vectors to be outgoing, they should satisfy the following 
conditions 

Vi + Vj + Vk = 0, ViAVj = l, {Vj AVk = VkAVi = l), (4.15) 

with Vi A Vj := fj,ifj,2 — Vi^2Vj,i. These correspond to the Calabi-Yau condition and 
the smoothness condition. Since the refined topological vertex has no cyclic symmetry. 
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we should specify a preferred direction. Therefore one of these three vectors should be 
the preferred one and we denote it by white arrow. Note that if we choose the middle 
edge as the preferred direction; Vj = (—1,0), then the condition fl4.15p implies that 
Vi = (a, 1), Vk = {b, —1) with a + b = 1. 




vertex edge 



Figure 4: Gluing rules 

Let {vi,Vj,Vk) and {vk,Vi/,Vj/) be the vectors associated with the vertices at the 
origin and at the end of the vector v^ of the kth edge, respectively. If we choose so that 
Vi and Vj are incoming and v^ and fj/ are outgoing, then the framing index of the 
kth edge is defined by 

Uk := Vi A Vi' = Vj A Vj'. (4.16) 

Each edge is associated also with a Young diagram A and a Kahler parameter Q G C so 
that the propagator for the kth edge is defined as 

Q^'/A.(g,t)"^ (4.17) 

and we glue the amplitudes by summing over the representation A on each edge. 



5 Four-Point Functions 

Here we show how to calculate the partition functions. The building blocks for them are 
the following four-point functions. 

5.1 Building blocks 

Assume that each vertex has a horizontal edge, which we take as the preferred direction. 
Fix the orientation of the preferred direction, say (—1, 0); then we have four possibilities 
of the configuration of two horizontal edges [Fig. 5]. Although the slopes and directions 
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of "vertical," i.e. nonhorizontal, edges can be arbitrary, we show in Figure 5 the simplest 
one whose internal edge is orthogonal to the preferred direction and we tentatively take 
the orientation of "vertical" edges from the top to the bottom. The framing index is 1, 
0, and —1, respectively. They are independent of the slope of "vertical" edges, but 
change the sign according to the orientations. 



II II II n 




V V V 



Figure 5: Four-point function: the framing indices of the internal lines are 1, 0, and 
— 1 from the left. 

We order the three edges at each vertex in the clockwise direction such that the 
preferred direction is in the middle position. This fixes the ordering of three edges 
uniquely. The lower and upper indices of the refined vertex correspond to the incoming 
and the outgoing representation. Then the following four-point functions are building 
blocks for the partition function: 

^mia/(Q; t) ■■= E ^MAi^?, ^)c,A/(g, t)Q\^^fr, (q, t) , 

Z^M^'^iQ; q, t) := Yl <^MAin?, t)C'''\{q, t)Q^'^, 

q^ t) ■■= E t)c,x,''{q, t)g'''i, 

V 

V^'^'^(Q; q, t) := Yl OQ'"'/. (q, ty' ■ (5.i) 

Note that 

Z^'^'^'^iQ; q,t)^Yl ^r,^^.^'^ ?)^-^A.v''^ {t, g)(-l)H + |Ail + |A.| + IHQl.l/^ ^f, q) 

= Z^.x.^M^'^ (g; t, (5.2) 



18 



We will show that Z^^^\,^^{Q] q,t) is related with Z^Ai'^^^(Q; g,t) by the flop. If we take 
the orientation of "vertical" edges from the bottom to the top, the sign of the fram- 
ing index changes. The corresponding four-point functions are written by C^xu{q,t)^s 
and Cfj_^'^{q,t)^s and they are the same as those in (15. ip up to the framing factors 
ffi iQ^t)"^^ fu {(l^t)'^^ for the outer "vertical" edges. 

Although we have flxed a preferred direction in this article, we can change it in some 
special cases. Let 

Z%/(g;g,t) := J2C.n'il^t)C-\{q,t)Q\^\ (5.3) 

V 

then from (14. 9p . we have the following symmetry 

^./•(Q; q, t) = Z\,^{Q- g, (g, t) /U (g, t) , (5.4) 

which changes the preferred direction. 

• ^^ 




Z.^'"{Q;q,t) ~ Z%.^^iQ;q,t) 

Figure 6: Changing the preferred direction 

5.2 OPE formula 

Next, we turn to showing some formulas for calculating the partition functions. Let us 
denote a symmetric function / in the set of variables {x\,xl,--- , , ■ ■ ■ , , x^, ■ ■ ■ ) 
by / (x^,x^, ■ ■ ■ ,x^) or / ({x*}^^). To calculate the partition functions, the essential 
part is the following Cauchy formula for the Macdonald function, 

J2 ^a(x; q, t)PAV (y, t, q) = no(x, y), (5.5) 

A 
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or, more generally, 

^Px/f^ix;q,t)Pxv/„v{y;t,q) = Uo{x,y)^Pf,v /xv {y;t, q)P^/xix; q,t), (5.6) 

A A 

with Ilo{x,y) in (I2.14p and the adding formula 

^Vm(2^; 9' t)Pf,/^{y; g, t) = Px/u{x, y, q, t). (5.7) 

Note that for c G C, Ilo{cx,y) = Ilo{x,cy), and for our involution l in (14. ip . Ilo{Lx,y) = 
no(x, Ly) = no(x, y)^^. Using these we have the following lemma. 
Lemma. Let x, y, z and w be sets of variables and a, (3 and 7 G C. Then 

Y ^M^Mv (x; t, g) P^/,^^ (y; g, t) P^v/^^v (2;; t, g) P^/^^ (^5 ^) a'''''/9''''7''''' 
= 5Z ^mV*?^ "/^^; ^) Pyh (/^72/, ^) (a/37)'''' no (y, /3z) . (5.8) 



Proof. Let a = a/6, = hj ^ = cj d\ then, from ( IB. lip . (15. 6p and (I5.7p . the left-hand 
side of the above equation is 

o-i,r),(T2 

= X] ^''M'' (~! ^' ^) Ai^/'?^ ^' ^) P'^ih (by; g, t) Pi./<X2 {dw; g, t) al^lrf-l'^lHo (^fey, ^) 

a"l,f;,o"2 

= 5Z A^/^^ ~5 ^' ^) A/'? (^^' ^' ^) a''''c?"''''no (by, 

= 5Z A^/'?^ ^) A/'? w^; 9, (^) no l^y, , (5.9) 

and the lemma is proven. □ 
Successively using this lemma, we obtain the following OPE formula, which is useful 
for calculating more general diagrams. 

Proposition. Let a;*'s be sets of variables, Cj^j+i G C and Cjj := Htl Ci,t+i. Then 

N 2N-1 

E n ^A2.-2v/A2.-.v {x''-'- 1, q) Px,./x,._, (x^^ g, t) J] c&ji 

{Ai,A2,--- ,A2jv_i} j=l i=l 

= XI ^AoV/r,v [{Ci,2i-lX^'~^}^=l\ t, q) Px^rf/V {{x^'C2i,2N}f=i; Q, t) C^^^^N 
V 

X J]^ Ho (X2i , C2i,2j-lX2j-l) , (5.10) 
l<i<j<N 
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for any integer N >2. 

Therefore the number of Young diagrams to perform summation reduces from 2N — 1 
to one. If Ao or A2Ar is the trivial representation, then since P,/\{x] q, t) = 6,^\, the number 
of Young diagrams for perform summation becomes zero. The trace over Aq = A2Ar is also 
calculated by the trace formula explained in the next section. If we realize the Macdonald 
polynomials by bosons as in [33], these OPE formulas come from the operator product 
expansion of vertex operators. 

5.3 Computations of four-point functions 

Here we apply the OPE formula (15.81) to the above building blocks. Let and be 
the set of variables as = q^°'t^ and y'^ = t^""^ q^, respectively. Then 

^mAia/(Q; g, t) = Px, {f- q, t) Fx, {f- g, t) U (g, ty' v-\''\ 
Z^x.^'^iQ; q, t) = Px, it'; q, t) Pa^v (-g"; t, g) 

x^p^v/,,v (-.yV,g) Am (x^g,t)P,v/.,v (V;t,g) Am t;i'^^i-i'^^iQi''i, 

g, t) = {-qP; t, q) Px, (t"; g, t) {q, t) U (g, i)"' 
xJ]P^v/,^v (V;t,g) Am P,v/,,v {-iy^-t,q) P,/,, (a;2;g,t)t;-l'^^l+l'^^lQl''l.(5.11) 

a'i,r),(T2 

From (15. Sp . they reduce to 

Z.x.x^'^iQ; q, t) = Pa, (t^; g, t) Pa, (t^; g, t) U (q, ty' v'^''^ 

X J2 ^AV'?- i-y'^ -Qy^-^ q) P,/^ {Qx\x^- g, t) {vQy\lio {-v-'Qx\ y^) ' , 

z^x.^^'^iQ: g, t) = Ai (t''; g, t) A^v (-g"; t, g) 

X ^A^"/'?" (-'^'' ^' ^) ^-/'' (^"'Q^'' 9' ^) ^''''no 1/2) , 

Z.^^^^iQ; q, t) = Pa,v (-g"; t, g) Pa, {fP; g, t) (g, t) U (g, t)"' 

X ^^^^^^ i-y'^ -v-'Q^y'; t, q) P.h {vQtx\ x^; g, t) {-Qx\y^) . (5.12) 

V 

Since Ilo{—Qx^,y'^)/Ilo{—Qt'',q'') = NxiX2 {'^Q; q,t), the instanton part, such as 
Z^.x^Xi'^iQ; q, t)/Z,„'{Q; g, t) is written not by IIq {-v'^Qx^, y^s but by A^AiA, (Q; g, ^)'s- 
Note that 

^,A/^^(g;g,t) = Z^X2''^\Q;q~\t-') = Z.va.v^^^^^ (Q; t, g)(-l)l^^l+l^^l+l^l+l'^l. (5.13) 
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5.4 Flop operation 

The flop invariance of tlie topological vertex is sliown in 
invariance of the refined topological vertex as follows. First 



We can show the flop 



mI+I 



Next, from fl2:33|l and fl23^ . we have 

Uo{-Qx\y')/Uo{-QtP,qP) 



,1 U,{-Qx\y') f,{q,t) 
Uo{-Q-'x\y^)U{q,ty 



Nx^\2 {vQ]q,t) 



Ho {-Q-^x\ 1/1) /Ho i-Q-HP, qP) N^,x, {vQ~\ q, t) 



Thus we obtain the following flop invarianc^ 



Q 



A1I+1A2I /a, iq,t) 

f\2 t) 



Z,'^x/iQ;q,t) ^ Z,^,^-'^{Q-^- q, t) ^|a,|+|a.|+|.|+|.| U (g. t) jq, t) 



Z^.\-\Q;q,t) Z,,,'^'{Q-^;q,t) 

The denominator corresponds to the perturbative part. 



^— A2 A^^^ 



Z,^\2'^{Q;q,t) ~ Z,,2^''^{Q-^;q,t) 

Figure 7: Flop invariance 



Combining (15.161) with (15. 4p . we have 



Z.%-iQ;q,t) 
Z.\'{Q;q,t) 



Q 



Z.,-'{Q-\q,t)U{q,t) ^ Z\.'^{Q-';q,t) 
Z..-iQ-\q,t) f,iq,t) Z'..'iQ-\q,t) 



which changes the preferred direction also. 

^ The flop invariance of C^^J^^^ {t, q) has recently been discussed in 



(5.14) 



(5.15) 



(5.16) 



(5.17) 
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Z.%'iQ;q,t) ~ Z./«(Q-i;g,t) ~ Z\.'^{Q-'; q,t) 

Figure 8: Changing the preferred direction II 

5.5 Finite N Macdonald polynomial and homological invariants 

When Ai or A2 is the trivial representation, the amplitudes of the above diagrams are 
written by the Macdonald polynomials with a finite number of variables. Note that 
Px {ax^, bix"^; q, t) with = q^^t^ and a, 6 G C is the Macdonald function in the power 
sum functions 

Pn (ax') + Lpn (bx^) = {a" - l) - 6" (q"^^- - l) } T^^"') + ! (5.18) 



i=l 



For iV e N and N > i{X), 



nN ^ 



1=1 1=1 

which are the power sum symmetric polynomials in variables. Therefore P\ [q'^t^, t^^^^] q 
is the Macdonald polynomial in variables {q'^^t^~^}i<i<N- On the other hand, from 
(EH) and (ESD 

Nx,{vQ-q,t)= W {l-vQq^'-H'-')= J] {l - vQq^-H'-') 

_ Px^iqP,vQq-P;t,q) _ Pxit^ ,v''Q-H-'^; q,t) 

Px^{q^:t,q) Px{t^;q,t) ^ 

N.,ivQ;q,t)= J] {l - vQq-'^^+^-H') = J] (l - t;- W^'^'"') 
(*j)eA {«,i)GA 
PxitP, v-'Qt-P; g, t) Pav (g^ vQ-'q-^; t, q) 



Px{tP\q,t) Px-{qP\t,q) 



Q\^\fx{q,t)-\ (5.20) 
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Therefore, some factors in Z^x-^^"^^ / Z,,** and Z^^^ / Z,* might be written by the 
Macdonald polynomials in a finite number of variables. For example, if and one of the 
Aq, (a = 1 or 2) are the trivial representation. 

When v^^Q = with N & N, they are written by the Macdonald polynomials in 
variables. These are candidates for the SU {N) homological invariants. 
Note that 

WA,,(g, t) := C.x'iq, t)t;l^l/. (g, t) = {f; q, t) {qH"; q, t) , (5.22) 
has a nice symmetry [15|(Ch. VI. 6): 

WA,.(g,t) = W,,A(g,t). (5.23) 
When t = q, Wx^^iQ, q) gives a large limit of the Hopf link invariants. 



6 One-Loop Diagrams 

Some one-loop diagrams which correspond to the trace of the vertex operators can be 
calculated by the following trace formula. 

6.1 Trace formula 

First, we have: 

Lemma. Let x and y be sets of variables and a, b and c := a6 G C. If |c| < 1, then 

A,^t A:>0 



expi-^ 

I n>0 



1 pn{ax)pn{-y) - c" 
n 1 — 



(6.1) 



Proof. As in [T5](Ch. 1.5), let F{x,y) denote the left-hand side of the above equation. 
Then it follows from the Cauchy formula (15. 6p that 

F{x, y)=J2 ^Av/^v (ax; t, q) Px/^ (y; g, t) (a6)l'^l 
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X,ti 

= Yl ^/^V^" ^' ^) ^A^A 9' ^) «''''&'^'no {ax, y) . (6.2) 

Therefore 

F(x, y) = F(cx, y)no (ax, y) = F(0, y) J] Hq (ac'^x, , |c| < 1. (6.3) 

fc>0 

But 

F(0, y) = 5^ Pva(z/; = E = 11(1 - c")-\ |c| < 1, (6.4) 

A A n>0 

and the lemma is proven. □ 

From the above lemma and (15.101) we obtain the following trace formula. 
Proposition. For G N, let x* = x^^"'"*'s be sets of variables, Aq = A2Ar, Cj_j+i = 
C2N+i,2N+i+i e C, Cij := ni=I and c := Ci,2Ar+i = Ci^i+i- If |c| < 1, then 

W 2N 



{Ai,A2,-,A2]v} «=1 «=1 

=nr:Wn n n. c«,_,cv-') 

fc>0 1=1 j=i+l 

=-p |- E ^r4^ |e E -^.2.-1^" (--'^■"^) - -"I I • (6-5) 

I n>0 I 1=1 j=i+l ) ) 

Proof. From fl5.10p and (16.11) . the left-hand side of the above equation is 

E ^•^V'?^ ({ci,2i~ix^'"^}^i; t, q) PxM ({a;^'c2i,2iv},=i; g, t) cf|U4'iv,27v+i 

A,r, 

l<i<j<N 

nrr / 2i 27-l\ YJ^O {W'C2i,2N+l}f=l,{Cl,2i-lC^x'^'~^}'jL^) 
no (x , C2.,2,-iX ^ ) [[ ^ _ ^fc+i , (6.6) 



here Hq {y^jfii) = U^iUjLi'^oix\y^)- Then the left-hand side of ([63]) re- 



duces to 



1 ^ 

n no {X'\ C2,2,_1X2^-1) ■ n YT^l n no C2,2^+2,_1C'=X2^+^^-^) 



l<i<jr<Af fc>0 ij = l 
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fc>0 l<i<j<N l<j<i<N 

(6.7) 

which equals to the second hne of ( 16. Sp . □ 
From this trace formula, we can calculate one-loop diagram if the loop does not 
contain preferred directions and also the framing factors cancel out. 

6.2 Examples for N = 2 and 4 

For an example of the trace formula for = 2, let 

= P,v/,,v i-tqP; t, q) P^M (t^; t, q) P,./^,. {-qP; t, q) P./., {d^- 1, q) t;l'^^H'^^lAl^lgl'^l. 

(6.8) 

Then from (16.51) with (ci,2, C2,3, 03,4, 04,5) = (i), A, v^^, Q) and (x^, x^, x^, a;"^) = {—iq^, t^, —q'^, tt^), 
it follows that c = QA and 



C2,3 C2,5 ] _ [ ^ c/t; 
C4,5 C4,7 / \ Q 



(6.9) 



and thus 

^ Hp (t^ -t;c'=+ig^) Ho (t^ -t;-ic'=+ig^) 1 - c'^+i ' ^ ' ' 



From (II. sp . we obtain 



Z2 = exp<^-> -{^—^ n , , n w{ c''}}. (6.11) 



n>0 



If we separate out the part Zl^"^* : = Z2 (A = 0) = exp { — J2n>o ^5"/ iiT-ii ^ — ^ ^ ) (9 ^ — g 2 ) ) | 
then := Z2/Zf is 



^i^^^t = exp 



^1 A" (Qn-u-)^Q--u-n ^ 

^ n 1 - C" t2 - t-2 V02 - 0^2 H 



n>0 



As we will see in section 7.2, this gives the equivariant Xy genus of the Hilbert scheme of 



points on C^. 
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- Q 



V 

Z2 



/i4, (^4 
—*r- 



fj'i, Qi " M /is, Q 



Z4 




Figure 9: Examples for the trace formula: the framing indices for the internal lines of Z4 
are all 1. 

For an example for = 4, let 



{fla} « = 1 
4 



(6.13) 



with /is = /ii. Then from fl6.5p with {c2a-i,2a, C2a,2a+i) = (v.v ^Qa) and ^) 
(t^, -tg''), it follows that 



4 i+4 



fc>0 i=l 7=i+l 



(6.14) 



where c = Q1Q2Q3Q4 and 



/ C2,3 


C2,5 


C2,7 


C2,9 






(Qi 


C4,5 


C4,7 


C4,9 


C4,ll 






Q2 


Cej 


C6,9 


C6,ll 


C6,13 




Qs 


\ C8,9 


C8,ll 


C8,13 


C8,15 


) 




\Q4 



(6.15) 



Thus 



Z4 = exp <^ 

. n>0 



where Qi+4 = Qi. 



ELi (Q» + QaQa+l + QlQl+lQl+2 + C") 

{t2 —t '2j{q2 — q 2 ) 



(6.16) 
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7 U{1) Partition Function, Xy Genus and Elliptic Genus 



Nekrasov's U{1) partition function, the Xy genus and the elhptic genus are reahzed by 
our refined topological vertex, as shown in |T3]. Since the diagrams for U{1) theory have 
trivial framing, the vertex in [H] and the improved vertex in the present paper give the 
same answer. 

7.1 U{1) partition function 

First, the U{1) partition function is written as follows. Let 

Z:=J2^^'^C,x'{q,t)C'\{q,t). (7.1) 
A 

Then 

A ^ 

= 5Zn^ ^(1 - g«Wt«W+l)(l - q-a{s)~lf-l{s)y C^-^) 
A A 

from (14. 3p . This agrees with the U{1) Nekrasov's formula Z™^* (^ei,A^;q,t^ in (12. 4p . 
Using the Cauchy-formula (15.51) we have 

— eXp < > „ ■ 



exp 



n>0 i,j 

1 »■ -L^l 



= JJ(l-At^-*g±(5-J-))±i, < 1. (7.3) 
7.2 Xy genus 

Next, the Xy genus is realized as follows. Let 

Z:=J2 Q''^' A'^'C.A'^(g, t)C'\iq, t). (7.4) 

A,i/ 

Then 

Z = Y, Q'^^'A^'^A (t''; q, t) Pav (-g^; t, q) P, (gV; g, t) P,v (-t^'g"; t, g) . (7.5) 
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Figure 10: U{1) partition function, Xy genus and Elliptic genus 



From (14. 3 p and (I5.5p we have 



If we separate out the part Zp^'^ := 'ZuQ^''^C„''{q,t)C'%{q,t) = Uo{-QtP, g^), which is 
independent of A, then Z''''^ := Z/Zp"'^ is from fl^TTUl) 



^AA {vQ;q,t) 



Nxxil;q,t) 

X sex ^ ^ 

This agrees with the Xy genus (20) of [35] with vQ = y, v^^A = Q^^^ and (g, t) = (1/ti, 
or (l/t2,ti). 

If our refined topological vertex had cyclic symmetry, then this Xy genus Z™^* would 
agree with Z^'^^ in section 6.2, and hence the following identity should hold 



A seA ^ ^ 



exp 



^1 A" (i-rg")(i-g'"g"; ^ 



. n>0 



A^g" (1 -t")(l - g-") 



From (lB.2ip . this is close to the Cauchy formula for the Macdonald functions in power 
sums Pn = il- t"g")/(l - t") and (-A)"(l - g-"g")/(l - g""), i.e. 

Vr-AV^i TT ^ ^ ^ ^ ^ „~a'(s).£'(.) 

A sex ^ ^ 
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: exp 



>^ 1 (i-rg")(i-g-"g") ^ 



n>0 



inst 



Although we have no proof for (17.81) . computer calculations support that 2'"^'^* = , 
which strongly suggests a kind of symmetry of web diagrams. See also the discussions in 
the recent papers [T8| [36]. 



7.3 Elliptic genus 

Finally, the elliptic genus is written as follows. Let 

Z:=Y. qI'^I Al^lQ?C^A'^(g, t)C"^\(g, t). (7.10) 

Then 

xP,v(-g^; t, q) ^P./M^t": Q, t) P,v/,v {-t^" q'; t, g)t;l'^l-l'^l. (7.11) 

V 

From (14.31) and the trace formula (16. 5p with (ci^2j C2,3, £3^4, 04^5) = {v,Q2,v~^,Qi) and 
(x\x^,a;^,a;^) = (-it^^'g'', gV, -t^^'g'', igV), it follows that 



■(1 _ gaWtn^) + l)(l _ g-a{s)-l^-^(s)) 
A A 

-p. no(-QiC^gV, t""g^) no(-Q2c"g't^ t'^^qP) 1 

no(-t;-ic'=+ig^t^ t^"g^)no(-^;c'=+igV, t^'g^) 1 - c^^+i ' ^ ' ^ 

with c = Q1Q2 and |c| < 1. If we factor out the A-independent part 

:= J2 Ql^'Q?C,/(g, t)C^%(g, t) 

^o(-glC^t^ g^ no(-g2c4^ g^) i 

no(-w^ic'=+it'', g/')no(-i)c'=+it'', g^) 1 - c'^+i ' ^ ' ^ 

then Z^""* := Z/ZP'^''* is from I^JO^ . 

^inst _ V- A V^l rr (^Qic^g,t)iVAA (t^Qac^ g, t) 
A Ar,,(c^g,t)iV,,(T;2c'=+^g,t) 
^ (1 - vQl+'Qlq-(^¥^^)+^) (1 - i;g^g^+^g"Wt^W+^) 
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(i - vg^+^g^g-'^w-H-^w) (1 - t;gJg^+^g-'^(^)-H-^w) 

^ (1 - Q'lQlq-(^)-H-'(^)) (1 - i;2g^+ig^+ig-«(«)-it-^W) ' 

This agrees with the elhptic genus (24) of [35] with gig2 = p, vQi = y, v^^A = y^^Q^^^ 
and (g,t) = (ti,l/t2) or {t2,l/h). 



8 SU{Nc) Partition Function 

Nekrasov's SU{Nc) partition function is also reahzed by our refined topological vertex, 
as mentioned in [HI. 



8.1 Pure SU{2) partition function 

The pure SU (2) partition function without Chern-Simons couplings is written as follows. 
Let 

^e,Ve2 t) ■■= E ^-Ai'^^' ^)^ma/(?, t)Q[% {q, t) 

= J2 it'; q. t) {q^HP; g, t) lP^v (^-t^^^; t, g) P,, {t^; g, t) {v-'Q^,2) '''' 

= Ho (-t;-^gi,2 g'^t^ t^'^q') ' Px, (tP; q, t) Px, (f; g, t) , (8.1) 
from (15. Sp . where Qa,/3 '■= e^/e^. The dual part is 

Z^ll''!: {t, g) = ^ C.,y\t, q)C,^,y{t, q)Q[%^ {t, q) 

V 

= E ^"''^(^' ty' (-i)i'^i+i'^'. (8.2) 

V 

Then, from (14. 3 P and (15. 5p . it follows that 

^■■=Y. <Ve'^ (^'^) ^e-e- ^) (AQl,2) '^'^^ ' /a, (?, t) //a. (?, t) 

' ■* '1 

^ Ho (-t;-^gi,2 g^^t", t^^'g^j^'no (-t;-^g2,i^^'"g^ g^^t'')"' 



Ai,A2 



^ (1 _ ga{s)^^(s)+l) (1 _ ^-a(s)-l^-<?{s)) 



" (1 - + (1 _ q-a{s)-\l-i{s)y 

If we factor out the A-independent part Z^""^^ := Z''' {q,t) Z'^'i _i (t,g), then Z 
^/^pert g^g^ggg ^j^j^ ^j^g 5^ (^2) Nekrasov's formula Z'^^^ (^ei, 62, A^; g, t j in (1^ . 
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inst 




Figure 11: SU(2) partition function: the framing indices for the bottom and the right 
internal hne are —1 and those for the top and the left internal lines are one. 

8.2 Pure SU{Nc) partition function 

The pure SU{Nc) partition function with Chern-Simons terms is written as follows. Let 

{/ia}a=l 0=1 

Nc Nc-l 

(8.4) 

with Qa,/3 = Ga/ep and /xq = I^n^ = 0- Note that ai = a^^ = 0. From the OPE formula 
f lS.lOp . we have 

-1 

Z^l::::i:: ^) = H {-v-'Qa,^ g'-t^, t'^^g') U Pa. (t^; g, t) . (8.5) 

a</3 o=l 

The dual part is 
with := Ba/ep. Then, using Aa^m in (12.221) . 

' = '^JVr' '1 



Ai,--,Aiv„ a=l 
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Ai,--- ,AjVc o</3 

X H'^^-'^^'^) n (1 _ ^a(s)^£(.)+l) (1 _ ^_a(s)-l^-^(s)) ' (^•'') 

with fio = fiNc = ^0 = ^Nc =0. If we factor out the A-independent part Z^'"'* : = 
Zl: {q, t) Z'U -1 (t, q), then Z^*^* := ZJZ^^'^ agrees with the SU{N^) Nekrasov's 
formula Z^^* (ei, ■ • ■ , e^v,, A; q, t) in 




Figure 12: SU{Nc) partition function: the framing indices for the horizontal lines are 
N — 1 —m, N — 3 ~ m,- ■ ■ ,1 — N — m from the top to the bottom. Those for the left and 
the right internal lines are 1 and —1, respectively. 



9 SU{Nc) with Nf = 2Nc 

The partition functions with fundamental matters are also realized by the refined topo- 
logical vertex as follows. Let 

{fla}C(=l a=l 
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X E ^M2._iv/,,^v (-t^-'g"; t, g) P^.^v (-g^; t, g) 6P^,„/.,„ {q''H'; q, t) 

Afc 2Nc-l 

JJ^k..-.l-k..l pj glA^^^^^, (9.1) 



with /io = fJ'2Nc = o"o = cr2Ar^ = 0. As ill the pure SU{Nc) case, from flS.lUp we have 

( _ c _ S3 \ A^c 

X n ('^'' ^' ^^-^ (-^'' ^' ^) • (9-2) 

The dual part is 

^A..._,v A,v ^^^^^ = ^ J]C-'^-iA.._,^^^_^(^^^)C'.,„_,^,/-(g,t) n ^?til(-l)"^"-^'' 

2iVc-l' '1 

{va\a=\ a=l 

(9.3) 

with Q^_^ = eye;^ and e'2^_i = e2„_i. 

When A2q for even integers la is a trivial representation, let 

/3=1 ^2/3 62/3-1 



In addition, let Z'-^^ := Z/Zp^"-* with Z?^"^* := Z''-;.:*„ „ _^ (g, t) Z'iU* {t, g). Then 

' ^ 2JVc-l''" 1 

{A2q 



^inst \ ^ 

,} n„</3 {Nx^x, {Qa,p; q, t) iV,^v,„v (Q'„^^; t, g)) riaii ^^A^.-.A^.., (1; q, t) 



j2 n^^^A2iv.iA.._,i 

{A2.-1} Ua<p {Nx^Xp {Qa,p;q,t)NxpX^ {Q'p,a^Q^i)Y~^^'"^ Ilil ^A2<,_iA2<,_i (1;?,^) 



(9.5) 

gives the SU (Nc) partition function with Nf = 2Nc. 
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A 

ei/e2 y!^i ^1 iyi\ei/e'2 



62/63 ^ z^2j e'a/eg 
< 



63/64 



64/65 ^'/i4 



62Ar,-2/62Ar,-l " f^2N^-2 



e2Af,-l/e2Ar,y/A^2Arc-l 
• ^ 



A3 



A 



A 



Ui^< 6^65 



'^2Arc-2" _2/62iV,_i 



^2N,-l\e2N,-l/^'2N, 
• 



Figure 13: SU (N^) partition function with Nf = 2Nc. the framing index for the longi- 
tudinal internal lines are all —1. 
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Appendix A : Proof of the Proposition in Sect. 



2.1 



A.l Combinatorial identities 

We have the following formula for the Young diagrams, which translates the summation 
in squares into that in lows: 

Lemma. For all integers A'^;^ > i{X) and A'^^ > i{fi), 

(1 - g) E = 5^ " ^^-^^ 

(ij)GA i=l 

(1 - g) E 'i''-'''^-' = E E E E ^"'^''t'-'- (A-2) 

\i=l j=i i=l j=i+l/ 



Proof. dAH) follows from 1^'^ = (1 - " l)- 

The left-hand side of f lA.2|) reduces to 

(1 - g) E E t''q^^~^'+' E = E E - 

i=l k=0 i=0 i=l k=0 

= E E (^^'~''' ~ 9^'-"'+' ) , ( A.3) 

i=l j=i 

which equals the right-hand side of flA.2l) . |--| 
From 

E q^^-''H^-'= E q^''^'t^'"+ E q^''^''+'t^+^-\ (A.4) 

l<j<jr<Af+l l<i<j<N l<i<N 

(1A.2P is rewritten as 

(1 - E ^^'"^'^''^''"'"^^ = - 1) E q^'^^'t^'' + 1 E {q~^' - ■ (A.5) 

{i,j)£fi l<i<j<N^ i=l 

Note that if t = g and A = /i, (lA.Sp reduces to the formula of the Maya diagram: the 
length from a black box to a white one or black one is (Aj — i) + (AJ — j) + 1 (the hook 
length) or (Aj — i) — {Xj — j), respectively: 

(ij)GA l<i<j<Nx l<i<Nx i<j<\i+Nx 
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By using (lA.ip . we have: 
Lemma. For all integers Nx > and A^a^ > ^i^'^)^ 



j=i j=i 

Proof. Similar to (lA.ip . for all integers Nxv > £(A^), 

^ (^1 - r^«") g^-i = (1 - t-') = (1 - ^"') Yl (A-8) 

Therefore, with (lA.ip . 

JVav Nx 

(1 - ^) E (1 - ^"'0 = (1 - E (1 - ^'0 ^'-^ (A-9) 



i=l 1=1 



□ 



In the power sum function (11.81) . (1A.7I) is written as 

(tt + (g5 (^t-^^g-p^Lg") =0, LgC. (A.IO) 

Note that if t = g, flA.7l) reduces to the formula of the Maya diagram: the black boxes 
and the white ones are at Aj — z + ^ and — (A^ — i + |) of the Maya diagram, respectively: 

i=l i=l i=^-Nx 

Hence Ei>i9^'"'^^ + Ei>i 9"^^^*"^ = = Q'^^il)- 



A. 2 Factors in Nekrasov's formula 

We have the following formula for the Young diagrams, which implies the equivalence 
among several expressions of Nekrasov's formula. 

Proposition. The following ri^[Li, L2] q, t)'s (z = 1, 2, 3) are all the same. 

vn\;{L,,L2,q,t):= Y q'^-'^h^^- {t^^ - L^) + Y 9^^'+'"^^^-^ (t"'^' " ^1) ,(A.12) 

(«j)GA (i,i)GM 
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vn 



2,+ 



vn 



(Li, W, q, t) :=pi (t-^'g-^ Lig^) p, {q'^-^, L,t^) - [ A = /i = ], (A.13) 



vn 



{U,L2-,q,t):={p^ [qh^, Ut-P)pr (g-^f^ Lat^) - [ A = = ]} - 



- t2 



_1 1 

q2 — q 2 



vn'^-{L,,L,-q,t):={p, (r^'g-'', (t'^'g'', L^g"'') - [ A = /i = ]} 



q2 — q 2 

1 

- t2 



(A.14) 



with V := (g/t)2 and Li, L2 G C. Here pi is the power sum function in fll.Sp and 
[ A = /i = ] 's stand for terms substituting A = /i = into the foregoing ones. 
Proof. It is clear that 

'^V (^1' ^2; g, ^)?^ = '^if (^2, Li;q^^,r'^)/v = n^;^^^ (L2, Li;t,q)/v, 
(^1' = ^^^"^(^2, Li; g"\ = riJ'v'^AV (^2, Li;t,q)/v, 



^Af(^i>^2;g,t)f = njf (L2,Li;g \t ^)/f = njv^;,v (L2, i^i; g)/^^- (A.15) 



3,±/ 



3,=F 



Therefore, it suffices to show n^^ = n'j^ 



n 



. First, applying (lA.lOp yields 



n 



Xfi ~ 



3+ 



Next, we prove that n^^ 



„3+ 



From (1A.2I) xt. we have, for all integers A^a^i > ^('^)! 



'EE-EE 

1=1 j=i i=l j=i+l 



By replacing g, t and A in flA.2p with 1/g, 1/t and /i, respectively, 

1 



(A.16) 



1-g 



Adding these two equations, we have 



'EE -EE 



q^r-Nti- 



(A.17) 



n 



l';(Li,L2;g,t) + Li ^ g-¥ + J] g^-V"^ 
{«.i)eAt («j)ga 

A'am+I Nx,, Nx^+l 

'EE-EE 

i=l j=l i=l j=l 

'EE-EE 

1=1 j=i 1=1 j=i 



1-g 
1 

1^ 



gXi-flj^j-i 



(A.18) 
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Thus, the following lemma with Nx — — Nx^ shows that n}^ — n^^. 
Lemma. For any integers Nx > £{X), > £{ij,) and A^^v > 



□ 



n 



2,+ 



iT'xi!iiLi,L2;q,t) 



=1 i=i 

(«.i)eA (i,j)et^ 

r Nx+l JVm Nx N^+1 

i=l j=l i=l j=l 



1-q 



(A.19) 



Proof. 

t~^nx^{Li,L2;q,t) 

/ Nx 
\j=l 



1 -t 



' N„ 



\-t 



'N„ 



_ 1 - Li 1 ^ L2 
-[A = /. = 0] 



i=l 



i=l j=l 



= (e (.^' ^) (e k-"' -^)*'-^)-'-^^ 
E«^--q^)fEV«.-^)-[A=.=o, 



JVa Nf, 



N^ 



Nx 



i=i j=i 



1=1 



□ 



Note that and are independent of A^a's, if they are sufficiently large. Let 
nxi^{Li, L2; q, t) := n';(^(Li, L2; q, t); then it satisfies 



nxt,{Li,L2]q,t)v = n^x{L2,Li]q ,t )/w = n^vAv(L2, Li; t, 



(A.20) 
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Let 



N'^^vQ, Li, L2; g, t) := exp i - ^ 9lyn^^^^^ g", I , (A.21) 

I n>0 J 



then we have: 



Corollary. The following A^{'^(Q, -^^i, -^^2; ^)'s {i = 1, 2, 3) are all the same. 



no(-g {t^'g^L^g-^}) 

no(-g {t^Lit-P}, {gP,L2g-n) ' 
no(-g {t-^"g-^Lig^}, {g-A't-^L^t^}) 

no(-g {g-^LigP}, {t-^Lat"}) 



(A.23) 



I,^ (vQ, L„ L2; g, t) - n(.g {t-,L2tP};g,t) 



n(t;g {r^'g-^Lig^}, {t'^"g'',L2g-''};r\g-i) 



A/1 



n (t;g {g-P, L^qP] , {qP, L2q-P] ; t-\ q-^] 



By letting Li and L2 = 0, we obtain six expressions of Nx^ [Q] g, t) in Nekrasov's formula. 
This completes the proof of the proposition in section 12. 1[ 



Appendix B : Formula for the Macdonald Symmetric 
Function 

Here we recapitulate basic properties of the Macdonald symmetric function [15]. 

B.l Definition for the Macdonald symmetric function 

Bases of the ring of symmetric functions in an infinite number of variables x = (xi, X2, ■ ■ ■ ) 
are indexed by the Young diagram, i.e. the partition A = (Ai,A2,---)5 which is a se- 
quence of nonnegative integers such that Aj > Aj+i and |A| = J^i'^i < -^^^ example, 



40 



the monomial symmetric function is defined by mx{x) — Xi"'-^^ X2''''^^ • • • , where the 
summation is over all distinct permutations of (Ai, A2, • • •). The power sum symmetric 
function px{x) is defined by 

oc 

P\{x) ^ Px,(x)px2(x) ■■■ , Pn(x) = (B.l) 

1=1 

We introduce an inner product on the ring of symmetric functions in the following man- 
ner: for any symmetric functions / and g, in power sums pxs, 

1 - 9 

{fip)\9ip))q,t f{P*) g{p) I constant part, PI — ^ I _ Qp ' '^^"^^ 

The Macdonald symmetric function Px = Px{x]q,t) is uniquely specified by the fol- 
lowing orthogonality and normalization: 

{Px\P,U = if A ^ (B.3) 

P\{x;q,t) ^ mx{x) + ^uxi_tm^{x), Uxu.eQ{q,t). (B.4) 

ij,<\ 

Here we used the dominance partial ordering on the Young diagrams defined as A > // 
|A| = and Xi + • ■ ■ + Xi > /ii + ■ — h //j for alH. 
The scalar product is given by 

{Px\Px)q,t = 11 ^ _ qa{s)fe(s)+l ' (^■^) 

which satisfies 



{Px\Px)q,t = (f ) {Px\Px)q-^,t-^ = (^av|Pav),7^ (B.6) 
If we define 

gxiq,t) :=———, (B.7) 

{^X\^x)q,t 

with V — (q/t)^, then 

9xiq,t) ^ 9xiq~\t-^) ^ 9x-it,qy^. (B.8) 
The skew Macdonald symmetric function Px/i_,{x; q,t) is defined by 

Px/^^{x;q,t) :^g^{q,t)P;{v-'x;q,t) Px{x;q,t), (B.9) 

where * acts on the power sum as := Uj^-^. Finally let lPx/^{x; q,t) be the skew 
Macdonald function with the involution l acting on the power sum p„ as i{pn) = —Pn- 
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Let X — (xi, X2, • • • ) and y — (yi, ?/2, • " " ) be two sets of variables. Then we have 

P\/^i{x] q, t)P^/^{y] q, t) = Px/u{x, y, q, t), (B.IO) 

where Px/t,{x,y; q,t) denotes the skew Macdonald function in the set of variables 
{xi,X2,--- ,1/1,1/2, •••)• 

B.2 Symmetries and Cauchy formulas 

Next, we turn to showing the basic properties of the (skew) Macdonald symmetric func- 
tion. The Macdonald function enjoys the symmetries 

Px/^^{cx; q, t) = c^^^'^^'^Px/^^ix; q,t), c G C, (B.ll) 

Px/^.{x;q-\t-')=Px/^,{x;q,t), (B.12) 

Pxytx^ivx]t,q) ^ ^^^p^Ug^tPx/„ix]q,t), uj^^tiPn) = (-1)''"^ ^ _ ^^ Pn- (B.13) 
When t = q, the Schur function has the extra symmetries 

sxv(x) = Lsx(-x) = (-1)\^\lsx(x). (B.14) 
The following Cauchy formula is especially important: 

Yl ~ i _ „ Pn(^)Pn(y) > 
n>0 ^ ^ J 

=nn i?i<MB.i5) 

1 — vxiy^q'^ 

k>0 1,3 

\n-l 



XI Pn{x)pn{y) 

n>0 



The Cauchy formulas for the skew Macdonald function are 

^ i)Px/u{y^ Q, t) = ^{vx, y) X] ^/./a(z/; g, t)Pu/\{x- q, t) ^^|^' 

XI -Pa/m(3^; t)Pxy/uy (y; g) = no(x, y) X] Pi^^/x'^iv; t, q)Pu/x{x; q, t). (B.17) 

A A 

If we denote by lu^^^ the endmorphism cu^^^ on variables x, then 

U{vx, y; q, t) = U{v-'x, y; q-\ r') = ujl^ujlgU{v-' x , y; t, q). (B.18) 
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B.3 Specialization formulas 

We denote 



p„(cgV) := c" y (g"^' - l)^^^') + c G C, 

^ t2 — t 2 



Pn{cqHP,cLt-P) ■ = pn{cqHP)+Pn{cLt- 



1 - L" 



c"y(g"^' -l)r(^-') + c"^ ^, c,LeC. (B.19) 

^ t2 —t 2 

1=1 



Then by using (1B.13I) and (1A.10I) . we obtain 



The Macdonald function in the power sumsp„ = (1 — L"')/(tt — t~t) is [15] (Ch. VI. 6) 

sex ^ 
for a generic L G C. By replacing (g, t) and A with (t,q) and A^, respectively, 



Then we have 



Note that 

iPx {t-P, LtP- q, t) = Pa {t', Lr^; q,t) , LeC. (B.24) 

If L = t"^ with N e N, then pn{q^tP ,r^-P) = ^^^^ g^^^r^^-^) is the power sum 
symmetric polynomial in variables {g^'t^^*}i<j<Ar, hence Pa (f, t^^~P; q, t) reduces to 
the Macdonald symmetric polynomial in variables. Therefore 

Pa (t", r^'"; q, t) = 0, for i{X) > N eN. (B.25) 

Note that 

WA,;.(g, t) := Pa (t^ r^-^; q, t) {qh'\ t"^-^; g, t) , iV G N, (B.26) 
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has a nice symmetry [15] (Ch. VI. 6): 

WxAQ,t)=W,,xiq,t). (B.27) 
When L = (the case of principal speciahzation) , 

Px it'; q, t) l[{~l)q-^W(^^ = P, {t-P- q, t) = P,v {-q^; t, q) /g,{q, t) 

= iPx (t"; g, t) = lPx^ {-q-P- 1, q) /gx{q, t). (B.28) 

Appendix C : Refined BPS State Counting 

From the instanton expansion of Nekrasov's partition function, 

oo 

+ 5^A%(g„;g,t) , (C.l) 

fc=i 

we can compute the refined Gopakumar-Vafa integer invariant as follows. We 

expect the following multicover structure of the partition function 

Z..^exp( yWS-^f^"-'") ). ,C.2, 

from the argument of Gopakumar-Vafa type. Assuming the scale parameter A is propor- 
tional to the Kahler parameter Qb of the base space of ALE fibration, we expand 

CO 

G{Qo^, Qb; q,t) = J2 QbG^{Q^- q, t) , (C.3) 

k=l 

where 

j^jUlJr) N~1 

and Xj{^) is the irreducible character of SU{2) with spin j. We have introduced the 
notations = t ■ q and = q/t. Comparing the coefficients of A'^ ~ Q^, up to = 4 
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we obtain 



Gi{Qa]q,t) = Zi{Qa;q,t), 

G^iQ^; q, t) = Z,{Q^; g, t) - ^ {Z,{Q^- q, t)f - \z^{Ql; q\ f), 

GsiQa; q, t) = Z^iQa; g, t) - Z2(Q„; g, t)Zi(g,; q,t) + ^ {Z^iQ^; q, t))' - ^Z.iQl; q\ t^), 

1 2 

G^iQa] q, t) = Zi{Qa] q, t) - Zs^Qa, q, t)Zi{Qa] q,t) - - {Z2{Qa] q, t)) 

+ Z^iQ^; q, t) {Z^{Q^- q, t)f-\ {Z.iQ^; q, t))' - ^Z^iQl; g^ t^) 

+ \{Z^{Ql;q\t')Y. (C.5) 

There is a cancellation of Zi{Q^] q'^,t'^) in the computation of G^^i^Qa] q,t). 

In 111] we reported some results for SU{2) theory with no Chern-Simons coupling. 
This corresponds to the refined GV invariants for the local Hirzebruch surface of Fq. For 
SU (2) theory the expansion at instanton number k becomes 

G,iQ,;q,t) = J2Y1 _ _ ^^y,. X,An)xUvV'Q',^'' , (C.6) 

where is the Kahler parameter of the fiber P^. The analysis of the symmetry of 
Nekrasov's partition function made in section 2 instructs us to factor out v'^^Qp in com- 
puting N^B+nF- results are 

N^^ip^ = S,,,oS,^,n^ , (C.7) 



for one instanton and 

n n-£+l 



m + 1 



1 3i + 2m 



(C.8) 



for two instantons. 

We have computed the invariants of SU (2) theory with the Chern-Simons coupling 
m = 1, 2, which are expected to give the refined GV invariants for local Fi and F2. It has 
been known that the GV invariants of Fq and F2 are simply related by a "shift" of the 
Kahler parameters. We have found this relation survives for the refined GV invariants up 
to instanton number 3. To describe the result neatly, let G'^^\Qf'-, q, t) be the coefficients 
of the instanton expansion (IC.3I) for local F^. Then what we have checked is 

G^^\Qr,q,t)=Q'p-Gf\Qr,q,t), (1 < < 3) , (C.9) 
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which imphes N^b+uF ^oca\. Fq is the same as 

N)^B+{n+k)F for local F2. We would 
like to stress this is a somewhat surprising result, since the refined GV invariants are 
not BPS protected quantities and they may jump under the deformation of complex 
structure^lj. For the GV invariants which are BPS protected, the agreement of the 
invariants may be explained by the fact that F2 is obtained from Fq by a deformation 
of complex structurqlj. However, for BPS nonprotected quantities it is not certain if the 
same argument applies. In any case what we have found supports the expectation that on 
noncompact Calabi-Yau manifold the refined GV invariants are actually invariant under 
the complex structure deformation, which is pointed out in [TT] . 

For local Fi the invariants are qualitatively different from local Fq at one instanton. 
We have 

N^Hf^ = 5,.,o5,«,n . (C.IO) 

For Fq and F2 the right spin at one instanton is always half-integer, while for Fi 
it is integer. However, at two instanton our computation shows that the refined GV 
invariants of Fi are related to Fq quite similarly to the relation between Fq and F2. We 
have checked that 

G^t)(Qp;g,t)=Q^G(°)(Q^;g,t), (A; = 1) . (C.ll) 

It has been pointed out that for even instanton number one may expect the GV invariants 
of local Fq and local Fi are related [7]. It is tempting to conjecture that the above relation 
is valid for any k. 

For general values of the Chern-Simons coupling, our preliminary computation shows 
that the refined invariants have no simple relation to those of local Fo,i,2- Even wrong the 
structure of Spin{4) character seems lost in this region. This may be related to the fact 
that the five- dimensional theory is physically not well-defined for these Chern-Simons 
couplings. 

For SU{3) case the computation of the refined invariants gets more involved. The 
corresponding local toric Calabi-Yau geometry is the ALE fibration of A2 type over 
and we have two Kahler parameters Qi := e~*^i and Q2 '■= e~*^2, for the fiber. The 

""^"^However, for local CY the deformation of complex structure may not be well-clefined, because of 
noncompactness of the total space. 

^^We thank Y. Konishi and S. Minabe for discussion on this issue. 
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instanton expansion takes the form 

oo J\jijL,jR) 

(C.12) 

where P{ni, 712) := kB + riiFi + 71,2-^2 represents two cycles wrapping k times on the base 
space. The analysis of the symmetry of Nekrasov's partition function made in section 2 
instructs us to factor out v^^{QiQ2)^ . At one instanton we found that the spin contents 
for the homology class B + niFi + 722-^2 are 

(0,nmax)©(0,nmax-l)©---©(0, |ni-n2|) , (C.13) 

where nmax '■= max(ni,n2). We note that the left spin always vanishes at one instanton. 
When ni = orn2 = the geometry reduces to local Fi and the above result is consistent 
with the refined GV invariants of local Fi. At two instanton since we cannot find any 
simple rule for the refined GV invariants, let us present a short list of our computation. 
When rii = or 77-2 = 0, the result is again consistent with ( IC.SI) in view of the relation 
flOTT]) . 
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{ni,n2) 


spin contents 


(1,0), (0,1), (1,1) 





(2,0), (0,2) 


(0,1) 


(2,1), (1,2) 


(0, 1) © (0, 1) 


(3,0), (0,3) 


(|,4)©(0,|)©(0,|) 


(2,2) 


(0, 1) © 2(0,1) ©2(0,1)© 2(0, i) 


(3,1), (1,3) 


(i,4)©(i,3)©2(0,|)©3(0,|)©(0,|) 


(4,0), (0,4) 


(1, f ) © (i, 5) © (i, 4) © 2(0, 1) © (0, 1) © (0, 1) 


(3, 2), (2, 3) 


(i,4)©(i,3)©(i,2) 

©(0, 1) © 3(0, 1) © 5(0, 1) © 4(0, 1) © 2(0, i) 


(4,1), (1,4) 


(l,f)©(l,|)©(i,5)©3(|,4)©(i,3) 
©3(0,|)©5(0,|)©3(0,|)©(0,|) 


(5,0), (0,5) 


(f, 7) © (1, f ) © (1, f ) © 2(1, 6) © (I, 5) © (1, 4) 
©2(0,^) ©2(0, 1)® (0,1)® (0,1) 


(3,3) 


(|,5)®2(l,4)®2(l,3)®2(l,2)®2(i,l) 

®(0, f ) ® 3(0, 1) ® 6(0, 1) ® 8(0, 1) ® 8(0, 1) ® 6(0, |) 
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Appendix D : (/-Dunkl Operator Realization for the 
Refined Topological Vertex 

In this appendix, we use the Macdonald polynomials P^ix', q,t) in the finite number of 
variables x = (xi, X2, ■ ■ ■ , xn) with setting xn+i = xn+2 = ■ ■ ■ = 0. Here we assume that 
|g|, \t\ > 1, and define the following refined topological vertex (without framing factor) 



N—*-oo '—^ ^ ' ' 
a 

lim ^P^^ .Pj), (gV; g, t)Pfv t, g)!;'-^! = iV,,\ (D.l) 

a 

These also reproduce Nekrasov's partition function. 

Let Yi, {i = 1, ■ ■ ■ , N) he the g-Dunkl operator [37] [38] acting on the variables Xj, 
(^ = 1,--- ,iV); 

Yi{x) = r-TiTi+, ■ ■ ■ Tn^iuT,-' ■ ■ ■ T-\, 

T. = tUH^^^^^(..-l), (D.2) 

i — Xi/Xi^i 

where 

Si = {i,i + 1), u = t^Sn^i- ■ ■ Si, TN^Xi) = q^^'^Xi. (D.3) 
They commute with each other, 

[Y,ix),Y,ix)]=0, (D.4) 

and the Macdonald polynomials are eigenfunctions of any symmetric operator / in them: 

- ,r;v(x))Pf(a:;g,t) =/(g^^t^\--- ,gM^^)Pf(a;;g,t). (D.5) 

Let Yi{x) be the dual g-Dunkl operator which is given by replacing g with t in Yi{x), i.e. 

fiY,{x), ■ ■ ■ , yiv(x))Pf (x; t, g) = /(t^^g^\ ■ ■ ■ , t^-g^-^)Pf (x; t, g). (D.6) 

Note that Yi{x) and Yi{x) may not commute with each other. 

Using these (dual) g-Dunkl operators, our vertices in fID.ip are written as follows: 



^MA'^ = i^il5Z^''''^'V<x(-^(^);?'^) ^,,t (6P,^v/,v(F(x);t,g)Pfv(x;t,g)^ 
y/'^= hm 5^t;Hp^^,/^,(-r(a;);t,g) , (6Pj;,(r (x); g, t)Pf (-x; g, t)) U=,p(D.7) 



a 
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Here tu^^ is the involution in (IB.lSp . 

Therefore the summation in the Young diagrams in Nekrasov's formula is formally 
performed by using these g-Dunkl operators. For example, the SU (2) partition function 
in (ESD is 

Z = Jhn^no (^-Q,Y{x),Y{z)y' Ho (-Qs^^H, 

Xllo{-Ax,y) Ilo{-Az,w)\:r=z=tP,y=w=qP- (D-8) 

In the SU (Nc) case, let 
Do:=nno(-Ax",2/"), 

a=l 

Da:= n no(-Q„,/3r(x"),F(x^)) no(-Q^,<.r(/),r(i/")) , 0<a<iV„(D.9) 

I3=a+1 

and D'^ := DaUJ^^u^^, then the SU{Nc) partition function in (18.71) with m = is 

7 \ ^ T/ MlT/ A'2 T/ /xjVc-lT/ • 

= -DjVc-1 ■ ■ ■ ^2^l^o\x'^=tP,y°'=qP- (D.IO) 

Since Lj^gU^ JIo{x,y) = Ilo{x,y), we have the following g-Dunkl operator realization for 
Nekrasov's formula 

Zq = Dnc-1 ■ ■ ■ D2DiDo\rca=tp,y°:=qp- (D.H) 



Appendix E : Notations and identities for Partitions 

For each square s = in the Young diagram of a partition A = (Ai, A2, ■ ■ ■ ), we define 

ax{s):=K-j, h{s):=X^-t, a'{s):=j-l, i'{s):=t-l, (E.l) 

where AJ denotes the conjugate (dual) diagram. They are called arm length, leg length, 
arm colength and leg colength, respectively. The hook length hx{s) and the content c(s) 
at s are given by 

hx{s) := axis) + his) + 1 , cis) := a'is) - f (s) . (E.2) 
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The weight |A| and ||A|p are 

i i seA 

We also need the following integer 

CO ^ oo 

n(A) J2^'{s) = - 1)A, = A^IA^ - 1) = J2^,{s) . 

seA i=l i=l seA 

Similarly, we have 

n(A^) :=^a'(.) = 5]a,(.). 

seA s6A 

They are related to the integer k{X) as follows: 

oo 

k{X) := 2 - z) = 2(n(A^) - n(A)) = |A| + J] A,(A, - 2z) . 

sex i=i 

Note that, since {A^ - = {J - ^}jU, 

(«J)eA (iJ)eA 
for any function /. Also since 

{A, - j}^u u {-/X. + J - = = {A. - u {-^^^ + 

we have 

E (^^ -^■) - E (/^^ + 1) = E (-^^ -^■) - E (/^^ + 

We list their relations: 
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\X\,\W,n,K 



Er i.i 



E 



|A| 



E.A. 



E(i,j)eA 1 



EseA 1' 



|A^ 



E,AJ 



i|l \l|2 



iEi Ai 



= E(i,,)eA(Ai - J + I) 



E..A + 1) • 



^IIAII^ 



1 \ V2 

2 l^j 



E(ij)eA(Ai ^ + L 



E.,A m + 1) , 



n{\) 


= E.(^ - l)Ai = 


E(i,i)GA(^ 


1) 






E.eA^'(«), 


2VM^ M 1^ \) 


= -Y" AV(A^ - 11 = 




- i) 
''J 






E.eA^(«)- 


n(AV) 


= E,0--i)AJ = 




■1) 






E.eA 


|(I|A|P-|A|) 


= iE.A.(A,-l) = 


E(i,j)eA(Ai - 


-i) 






E.eA«(«)- 




= iE,A,(A, + l-2i), 










E.eAc(s), 


n(A^) -n(A) 


= E.i(Ar-AO = 


E(i,j)eA(i - 






= E.eA(«'(«)-n«)), 


i(iiAii^-iiA-in 


= i Ei (Ai^ - Aj^^) = 


E(ij)eA(Ai - AJ 


+ i- 




= E, 


,,,(a(.) -£(.)). 


n(AV)+n(A) + |A 


HE. {^-\) + 


E(i,j)6A(^ + i 


-1) 






a'(s)+f(s) + l), 












sex 




|(I|A|P + ||A^IP) 


= i Ei (Ai^ + Aj^^) = 






-1) 


= E' 


{a{s)+e{s) + l), 












seA 






= iE^A,(A,-l + 2i), 










eJU)- 
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